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ABSTRACT . , _ 

The second year of an individualized multimedia 

bilingual magnet school project for non-English speaking background 
students in Philadelphia was evaluated. Approximately 275 students 
participated in programs at two > sites^ Bilingual multimedia learning 
centers containing materials for individual and small-group _____ 
instruction and computer terminals for computer-aided instruction 
were formed at each site. Students from Korean, Spanish, Portuguese, 
Chinese, French, Italian, land Vietnamese speaking backgrounds were 
provided bilingual instruction as well as instruction iivEriglish as a 
second language. Improvement in English aural comprehension , English 
reading scores, English speaking skills, Spanish reading scores, 
attendance, and retention in school were comparable to or greater 
than those of comparison groups. (Author/RW) 
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EXECUTIVE SUMMARY 



PROJECT DESCRIPTION 

An Individualized Multi-Media Bilingual Education Magnet Model served "limited- 
English-speaking-ability 11 pupils from a variety of linguistic backgrounds. At 
the elementary site, pupils whose mother tongue [s Korean; Spanish or Portuguese 
participated in bilingual instruction. At the secondary site, pupils whose mother 
tongue is Chinese (Mandarin or Cantonese) , French, Italian, Korean, Spanish, 
or Vietnamese were In bilingual Instruction. English for speakers of other lan- 
guages was provided for pupils who_were not proficient in .English; A Bilingual 
Multi-Media Learning Center (BIMLC) was begun at each site. This center contained 
a wide variety of materials for Individual and small-group instruction of pupils. 
At the secondary site, the BIMLC contained terminals used for computer-aided and 
managed instruction. 

KEY FINDINGS 

After one and one half years of operation, all pupil pefdrmance objectives 
were attained or considered attained. The rate of acquisition of aural comprehen- 
sion exceeded the rate found among pupiJs of the comparison group—pupils in the 
Title I ESOL program. 

Pupils in the program acquired speaking skills at the same rate as those in 
the comparison group. English reading scores increased at a faster rate than 
predicted from test-publisher's norms. Elementary school Hispanic pupils equalled 
or exceeded Spanish reading test norms for rural Puerto Rico. Secondary school 
Hispanic pupils 1 average score was equivalent to the 36th percentile of Puerto 
Rican tenth grade norms. Objectives regarding high school pupils 1 completion of 
the school year and attendance were attained. 



INTRODUCTION 



Bilingual-bicultural programs are usually designed to meet instructional 
demands which arise when a school population is composed of two groups, native 
English speakers and speakers of one other language. When there are more than * ; 
two groups, bilingual education is often avoided because the number of subgroups 
consisting of pupils sharing a common first language, a common level of experience 
with English, and a common age or grade level becomes inordinately large. The 
Individualized Multi-Media Bilingual Education Magnet Model program is an attempt 
to deal with this diversity in a cost-effective way. 

RATIONALE 

Bilingual-bicultural programs are intended to provide support to limited 
English-speaking ability pupils through use of English and the home language 
for instruction. The Individualized Multi-Media Bilingual Education Magnet Model 
program was designed to explore the feasibility of providing bijingual-bicultural 
instruction in a setting serving severa I language groups. To accomplish this 
goal, the Bilingual Individualized Multi-Media Learning Center (BIMLC) concept 
was developed and implemented at two sites: Birney Elementary School and South 
Philadelphia High School . At the elementary school a bilingual program was 
available in Spanish, Pc tuguese and Korean. At the secondary site a bilingual 
program was available i i Spanish; French, Italian, Vietnamese, Chinese ^Mandarin 
and Cantonese) and Korean. At both sites, pupils who spoke other languages also 
participated . These pupils made use of English to Speakers of Other Languages 
(ESOL) curriculum and relevant materials housed in the BIMLC. As some teachers 
are multi-lingual, many members of the smaller linguistic groups received Informal 
assistance through their mother tongue when needed . Tutoring supplemented 
classroom instruction for some students. 

'."17" * 

EXPECTED OUTCOMES ; 

Pupils are expected to make a smooth transition from instruction in their home 
language and ESOL to mainstream instruction at the school . In the course of 
making the transition, competence in the home language is to be enhanced or 
retained and growth in academic skill areas is to be maintained. As the^ BIMLC _ 
provides instructional resources which supplement ESOL, acquisition of English 

should be facilitated. 

At the secondary level, the use of the mother tongue and flexibility in course 
roster ing is expected to result in good attendance and more students remaining 
In school until graduation. 

PREVIOUS FINDINGS 

Last year, 1976-1977, the project was funded for the first time. The grant- 
award was issued later than expected, preventing management from implementing 



the project according to the schedule in the proposal . The program was opera- 
tional by mid-year and most enabling objectives had been attained by March. 
Evaluation of pupil performance during the spring suggested that the program 
accelerated pupils' mastery of English comprehension. Retention of pUpMs 
in the high school component was very good (96.2%) . Other data collected (Spanish 
reading, English reading, and attendance^ Vvere difficult to interpret because of 
the lack of pretests and because of ongoing implementation. 

PROGRAM IMPLEMENTATION 

Some aspects of the program implemehtation process were stated as enabling 
objectives. These aspects will be discussed In the next section. Overall program 
implementation is in this section. 

Nearly all elements of the program were carried out in a manner consistent « 
^vith the proposal, though differences in ages of pupils in the target groups, and 
organization of the mainstream program at the two sites resulted in programs 
which differed in many ways. 

Elementary Level . The birhey Elementary School program served pupils 
in grades one through four (Fifth and sixth grade pupils no longer attend the 
school.) . The primary language groups were Korean, Portuguese and Spanish, 
though speakers of other languages participated in some program ^elements : When 
-evaluators visited the site/ they found a highly articulated program which seemed 
to be a good adaptation of the overall plan for the local situation. 

In late Spring, 1978, there were 98 pupils in the elementary program. The 
site coordinator confirmed the evaluators 1 data which showed that the pupil 
population was quite stable. Most pupils who began the year completed it. v ; 

The school organization psrmitted all pupils to be assiged to regular^ main- 
stream home room classes for at least two hours of the five hour instructional day. 
The largest group of participants (71 pupils) was engaged in all program activities. 
These pupils studied in ESOL classes incorporating use of the Birney school BIMLC 
on a regularly scheduled basis. They studied their home language (Korean, 
Portugese, or Spanish) and parts of the elementary school curriculum in classes 
conducted in their heme language. In grades 1 and 2, mathematics and social 
studies were the fecus of instruction in the home language. In grades 3 and 4, 
science was also studied. 

Classes aimed at mastery of English, including those in the BIMLC, were 
comprised of pupils of heterogeneous language backgrounds. When the home 
language was the medium of communication, all members of the class were speakers 
of one language. v 
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In addition to the large group there were two smajjer ones. The first consisted 
of 13 pupils getting ESOL support, nine in the full ESOL program and four who 
made use of materials in the BIMLC. Many of the thirteen pupils were not speakers 
of the three main languages of the Birney program. Those only using the BIMLC 
were reported to Be able to work in the English majhstream^ buLwere in need of 
individually prescribed assistance. The second group consisted of fifteen pupils 
who were competent in English. They were permitted to continue studying their 
mother tongue. 

When pupils were observed using the BIMLC, at least two and often three 
adults were present — one or two teachers and an aide. They arranged for pupils 
to work Independently or in small groups, depending on the particular need of 
the pupil. The teaching equipment of the BIMLC was being used when evaluators 
were present. 

•The site coordinator reported that "three or four" pupils were unable to main- 
tain the self-control needed for the independent work being done in BIMLC. These 
pupils were temporarily reassigned to mainstream classes while their classmates 
were Using the BIMLC. 

Secondary Level . At South Philadelphia High School, the secondary site, 
students were older and a greater variety of languages were spoken. As a result 
the program differed from the elementary one in some instructional aims and in 
organizational pattern. 

Evaluators were able to obtain enrollment records for 170 pupils who attended 
the program and did not transfer to other schools or school systems during the 
course of the year. As expected in a magnet program for in-migrants and immi- 
grants^ the amount of time these students were on roll varied greatly. Ninety-eight 
(57 . 6%) were on rol I for 1 72 or more days of the 1 82 days of the school year . The 
remaining students were on roll for shorter periods of time. Pupil records suggest 
that pupils were admitted every few days. The shortest enrollment at .the site was 
thirty-nine days. 

Since the secondary level program served both students of typical high school 
age and older, grade assignments needed to be more flexible than at the elementary 
site. Depending upon a pupil f s experience in United States mainland schools, the 
pupil was assigned to one of three ungraded advisory (home room) classes or 
assigned to a graded mainstream advisory. During the course of the year, students 
in the ungraded advisories were assigned to grades on the basis of their school 
experience and academic achievement. The grade assignments were to be used for 
increasing the mainstreaming of students in the fall of 1978. 

While each student had an individual schedule of classes, those in the ungraded 
advisories received more program services than those who were in the mainstream 
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advisories. Pupils with the least competence in English attended more periods of 
ESOL instruction than di'd others, jf a pupil's first language was one of those 
offered at South Philadelphia, and ESOL was no longer needed, the r*jp\i might 
study all subjects in the mainstream, but still receive support through one class 
per day in the home language. 

Instruction In Spanish, French (primarily for South-East Asian and Haitian 
students), Italian, Vietnamese, Chinese (Cantonese and Mandarin) , and Korean 
was provided; Though this instruction emphasized mgstery of the home language, 
-concepts of other academic areas were* i ncluded . At times, teachers wpqld se^ve 
as tutors when students had specific su^^ts'thfy vyerjer " 

studying in English. In contrast to the elementary school, there were no courses 
for teaching mathematics, science, or social studies -in the pupils 1 mother tongues; 
A course in American history for foreign-born students, conducted in English, 
was available for the students. 

The BIMLC, which contained computer aided and assisted materials, was pri- 
marily used to reinforce ESOL. As it also contained English] language materials 
to teach mathematics and career planning^ it was possjbl^ T to reinforce special 
vocabulary and language structures found in these areas and their content. 

During the first part of the year, ESOL teachers used the BIMLC when they 
felt it was desirable, without following a set schedule. According to the site 
coordinator, by the end of the school year teachers were sending pupils to the 
BIMLC according to a schedule. Small groups of students were sent for individual • 
attention. The instructional material ised was specified by the teacher. The site 
coordinator and the BIMLC classroom aide ^arranged for Jttie student to use it . 
When evaluators visited the center, students were observed using the computer 
equipment as well as other materials. Some were working individuaiiy, others 
were in small groups of three or four students. o 

As the computer based instruction is a unique aspect of the high school <BIMLC, * 
and reading (in English) one of the more critical [h^structiohal areas, evaluators 
gave the Computer Aided Instruction (CAI) Reading Program special attention. 
CAI reading consists of an introduction and seventeen topics ranging in complexity 
from "Sentence Recognition 11 to "Content for Meaning . 11 Each topic begins with a 
-pretest, contains a sequence of developmental lessons and ends with a posttest. 

On March 14, the BIMLC aide interrogated the computer memory to determine 
the status of students. The results indicated that there were problems! The 
report, called a PRD, listed the topics each student had worked with, a pretest 
-score, a developmental score (reflecting work on developmental lessons) , and a 
posttest scc:-e. If the student passed the pretest, the student was to move to the 
next topic. If the student failed, developmental lessons were prescribed by the 
computer until a criterion was reached . Then a posttest was administered. . 





The PRD suggested that the system was misused or had malfunctioned . As 
of March, 137 pupils had used the program. While one pupil had worked through 
five topits, the majority of pupils (105) had worked with only one. Many records 

showed pretest scores of zero. All but two, pupils had developmental scores of 

zero, and all posttest scores were zero. As the computer printout did not indicate 
that the zeros were test failures, evaluators,do not believe they were really students 1 
scores. Discussion with a staff member of the Division of instructional Systems * 
and the aide in charge of the secondary -jchooJ BIMLC suggest that the problem 
may have been failure of students to operate the equipment properly. The aide 
also said the high school's US minute instruction periods were not long enough 
^ for iimited-English-speaking-abiiity pupils to work through a substantial part of 

a program or test in English. * 

In the course of evaluating product objectives, a second concern emerge. 
It became apparent that movement of students in and out of the ESOL classes at the 
secondary site was greater than ahticjpated. Whije evaluators expected to find a 
substantial number of new students admitted during the school year, they also found 
that many students had transferred out of ESQL. The movement of students out of 
ESOL became apparent when it was noted that many who . took auraj comprehension 
and reading pretests did not take posttests . This matter was reviewed with the 
site coordinator, who said that there was pressure^ from outside the program, 
* on students to drop ESOL in favor of mainstream English classes^ The project 
director made it clear that this was not consistent with program policy and was 
to be stopped, Evaluators had the impression that the director would take steps to 
relieve the pressure on students. As a result, more students should complete a 
full year of ESOL in 1978-1979. 

The high school BIMLC contained books in languages other than English. 
According to the site coordinator the books were resource materials for use when 
; needed by students and teachers . They included both language-arts materials 

and books in other subject areas. Thus, at the secondary level, the BIMLC con- 
tained some materials which could assist the student through the use of the home 
language. These resource materials are important in acprogram where many 
students study science, social studies, and mathematics in mainstream classes, and 
teachers play a tutorial role. 

In conclusion, it is the evaluators 1 judgement that the programs implemented 
at both the elementary and secondary sites were cohsistent^with the intent of the 
program proposal . At the secondary site, the supervisor and site coordinator 
should take steps to reduce movement of students out of the ESOL classes in mid- 
year and assure that thus, computer is used correctly. 
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;.- During 1977-1973* program staff will continue to identify, evaluate 

and purchase instructional materials and equipment for the Bilingual Individualized 
Multi-Media Learning Center. This will include: 

' . _ 9 :: : 



^Acquisition of appropriate grade-level instructional materials for teaching 
reading and for other subjects in each of the languages of the program . . 

.Reproduction and delivery of off-line materials needed for the cbmpuier- 
managed and computer-assisted instruction at the secondary School sites. 

Acquisition of other programmed instructional materials. 

This objective was attained. Documentation provided by projeot personnel 
confirmed that curriculum materials were being acquired, and observation of 
classes confirmfifTthat they were being used. 

* Eva I ua tors had an opportunity to review requisition (order) forms used to ; . 
purchase materials and equipment. Site coordinators supplied Jists of instructional 
equipment and materials oh order and on hand. The documentation reflected l} 
steps taken during the Spring of 1977 for the program year 1977-1978. 

Review of these documents showed that most of the "hardware" (e.g. , Borg- 
Warner System 80 programmed learning sets) was for individualizing the instruction 
in English . Books and other instructional* aides were in both the home languages 
of the pupils and in English and could be used for teaching language arts or other 
subjects. Site observations confirmed that equipment and texts were present at 
the schools. 

Interviews with site coordinators revealed that materials and I equipment were 
often delivered later than expected, limiting their usefulness during the year for 
which they were intended. However, the late delivery guaranteed their avail- 
ability at the beginning of the following school year. In these interviews the site 
coordinators reported that, when asked to "plan early" for the next program they 
were unsure of the number of copies of materials >vhich are designed to be kept 
by pupils (e.g. , workbooks) that would be needed. Site coordinators also reported 
that grade-level appropriate materials income languages (especially Oriental) . 
were unobtainable. Curriculum development was Initiated to remedy this problem.' 

Objective 2: A recruitment program will be conducted by project staff, district 
superintendents, principals of the two sites, and the Office of Informational Services 
that will attract hoh-Engiish-domlna^ Portugese. Korean, Chinese, 

Vietnamese, Italian and French-speaking pupils who live outside the regular 
boundaries of the project schools, thus creating the magnet program . This 
recruitment will begin in August, 1977 and continue through December, 1977. 
It will include: 

.News releases sent to newspapers serving each of the language groups. 

.Announcements on radio programs serving speakers of the languages of the 
program. 
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.Pupil recuriiment by members of We program's pafenfydvisory group. 

.Notification to all principals of elementary and secondary schools describing 
the new programs and asking them to identify students who can benefit from 
, participation in thernT~*^ 

• . „- _ 

This objective was considered attained. All four steps were taken to recruit 

students, but not all were carried but in all the languages of the program. There ' 

were also differences observed between the two program sites; - 

, _«t ... __»___ 

Evaluators obtained reports of newspaper acticles in the Viet Hong (the ViethSrhei 
Christian Newsletter)^ the Asian Amerie^^44wsletter , a local Korean newspaper 
and) Spanish press. To increase the-vfsibility of the pfbgrim at the secondary level, 
teachers prepared fliers in each of the target languages. According to the site 
coordinator, these fliers were distributed by students to shops around the city 
catering to different ethnic groups. They were a_tso given to sbcjaj agencies, ip- : 
eluding the Catholic Social Service Agency, Jewish FamTI^\Services, Nationalities 
Service Center, and Lutheran Children and Family Service^ The high school also 
houses the Foreign Language Magnet program of the School District. News releases 
and articles about the, Foreign Language Magnet program mentioned the Individual- 
ized Multi-Media Bilingual Education Magnet Project. 

During 1977-1978 there were announcements on radio and television coverage. 
Two members of the Foreign Languages division of the ^ School Djstrict had weekly 
radio programs in Spanish, during which the project was discussed. Coordinators 
reported that there was a radio program in Korean about the elementary site* The 
secondary site coordinator was interviewed oh television hews arid on radio. 

The School District has a bilingual parents advisory group, which met monthly 
and Included speakers of all languages serviced by the program^ Members of the 
group were asked to carry informatipn^bout the project to their communities. 

As part of the on-going provision of service to Ilmite^Ehglish-speakirig 
ability pupils, a city-wide screening process was conducted. When the screening 
was undertaken, district superintendents and principals were informed of the 
availability of bilingual education through the individualized Multi-Media Bilingual m 
Education Magnet Project. The secondary site coordinator and the elementary site 
principal described similar processes being used when pupils wished Jo Renter the 
project. The principal of the sending school contacted the site coordinator or 
principal, the appropriateness of the program for the pupil was determined, afid 
the district superintendent effected the pup! I 's transfer . The coordinator and 
the principal both felt that principals of other schools knew about the program . 
The elementary site principal reported that he had discussed the program with > . 
/ other principals at district staff meetings. 
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In addition to the steps listed in the objective, individual efforts were made, 
especially jay staff at the elementary site. The coordinator arid the teachers 
reported making visits to families speaking the language they'taught. In order to 
familiarize parents with the program. 

The student recruitment activities resulted in different outcomes at the two 
sites. At the secondary level, there was a true "magnet" program. The site 
coordinator reported that only 50 of the approximately 180 pupils served resided 
in the school feeder area. 

The elementary site principal reported tfiat most of the pupils resided in the 

school feeder _area; but Lsome Korean-speaking families had moved Into jt because 
of the program. He noted that Hispanic families did riot need to move because 
other Spanish-English programs served their children and that Portuguese 
families did riot move into the feeder area because their desire for parochial school 
education outweighed parents 1 interest in a bi Ungual program for their children. 

Objective 3: Program staff speaking Chinese, Italian, Portuguese, Vietnamese, 
Korean and French will, with the assistance of supervisors and evaluation personnel , 
develop informal reading inventories to be Used in subsequent years to evaluate , 
mother-tongue reading performance of pupils speaking these languages. These 
inventories wil.1 be prepared by June, 7975. 

' This objective was not attained . Acquisition and development of new eurricu- 
iurrumateriais in these languages occupied personnel (see Enabling Objective 1) , 
/preventing them from giving attention to test development. In following this 
.course, they were tin accord with priorities of the project management. 

* The project director stated that this objective continued to be a goal. In 
1978-1979, an attempt would be made to develop the informal reading inventories. 

Objective 4: Monitoring will show thai all staff and students are familiar with 
the use of the individualized materials and equipment in the BIMtC. 

This 9bjectiye wai considered aftairied-. Both sites were visited by program 
eva I uator-s*. extensively during the fall and spring. During the fall some teachers 
were not aware of all the resources of 5 the BIMLC arid sortie materials and equipment 
had not arrived. . During the spring bbservatioris, virtually all ordered material 
had arHyed Irid the teachers and students were using it. Sit^coordinators and 
aides appealed to have no difficulty iq using most of the BIMLC resources, and 
students appeared to know how to use the equipment, two problems rioted by 
evaluators were use of the computer terminals at the secondary site and use of 
the Voxcom at the elementary site. The Program Implementation section of this 
report has already described the difficulty with the computer . The Voxcom, a 
device for associating spoken words with writirig or pictures had not been used, 



according to the elementary school site coordinator , because it requires extensive 
planning by teachers. Evaluators believe that problems associated with use of 
the computer terminals should be monitored by supervisory personnel . The prob- 
lem with the Voxcom was regarded as minor and likely to be resolved by the 
elementary school staff. 



Objective 1: The rate of acquisition of English speaking and comprehension ' skills 
of pupils r, as measured by the Philadelphia ESOL Screening Test (Paris 11 and Hi) 
and the Test of Aural Comprehension, will be increased to a statistically significant 
degree (p<. 05) . 

This objective was attained. Pupils with limited English language skills who 
took the Test of Aural Comprehension (TAG) in May knew more than they had known 
when tested half a year earlier. The improvement in comprehension was even 
higher than in the Title I ESOL Program. This difference in the rate of growth 
was significant and substantial. 

Only 85 project pupils could be identified as having been tested twice during 
the year. These pupils, when tested in November _pr December, knew an average 
of 17.47 items. Prior analyses of over a thousand Title I ESOL pupils suggest 
that ESOL pupils tend to improve at the rate of .19 items per month (.68 attributable 
to exposure to ah English-speaking environment and .11 to the effect of participation 
in ah ESOL program) . Since an average of 169 days elapsed between the tests 
at the Individualized LMuiti-Media Bilingual Project sites, the project pupils were 
expected to gain 1 . 07 items if their rate of language acquisition was that of their ; 
Title! peers. Thus project pupils were expected to knSw ah average of 18. 54 items 
at the time of the posttest in May if participation in the program was no more 
effective than being in the regular ESOL program. When retes ted, the 85 pupils 
knfew an average of 22.88 items. The gain of 4.35 items above the average expected 
score was highly significant (t=_4.42* p<. 01). - 

in concluding that the program has had a substantial effect oh language compre- 
hension, evaJuatbrs used estimates of the effects of time from a sample of Title j 
schools. Though these estimates were shown to be precise enough for evaluating 
Title VII pupils (an error of one-third of an item is very unlikely) , the appropriate- 
ness of comparing these two groups may be questioned. After all* the difference 
between the two groups in the spring may hot be due to the program but to other 
extraneous factors. Four alternatives have been examined. Evaiuators believe 
that none can explain the superior performance of the project participants. 

.Differences may be attributable to economic background especially since one of 
the sites was not funded byTitie I ESOL. Evaiuators believe that the eco nomic 
backgrounds of Title I and Title VII pupils is comparable, because both sites 
met the eligibility criteria of Title ( funding and bcflh have had the Title I 
ESOL program in the past. ■ _ \ 
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.Differences may be attributable to ethnic background since 90% of the Title I 
pupils were Hispanic and only 20% of the Multi-Media sample were. Because 
non-Hispanic pupils may have been different in initial ability, the observed 
growth in score may be due to ethnicity rather than program effects. To test 
this hypothesis, the average score of Hispanics on the early administration of 
TAG was compared to that of Non-Rispahics. When other relevant variables 
were controlled there was no significant difference at the .05 level between 
Hispanic and non-Hispanic pupils. 

.Differences may be attributable to an interaction between ethnicity and'the 
effect of time on language acquisition (e.g., non-Htspanic pupils may learn 
more rapidly than their peers.) To test this supposition the 85 pupils were 
divided into three language categories: Spanish, other European languages 
(e.g., French, Italian, Portugese), and all others (Korean, Chinese, Vietnamese 
etc.) . A Duncan jnultiple range test comparing the average improvement over- 
expected score did hot reveal any significant differences among the language 
group.*. The 12 individual languages were also compared witfi each other ._ 
They did not differ significantly at the .05 level . The ©valuators concluded 
that there was ho difference in the rate of aural comprehension growth among 
the language categories. Since there was no significant difference among the 
groups either bri their initial scores or their gains, ©valuators concluded that the 
apparent effect of the program was hot due to an interaction between ethnicity 
and the effect of time. 

.The model specified for Title I ESOL pupils may have been correct for the 
ESOL pupils but. incorrect for the Individualized Multi-Media Bilingual Educa- 
tion Project. To test this hypothesis, the same variables Used in the Title I 
analysis were regressed on the gains of project pupils. None of the 9 variables: 
test form, sex, grade (2 trends) , place of birth (2 variables) , time in ESOL, 
time oh mainland , and language was statistically significant. The overaU F-test 
was likewise not significant. The jnultiple correlation between gains and the 
the other variables was small (.157) . Thus the estimation process, which used 
initial scores and expected gains to yield expected scores, seems to > have 
controlled for the background and time dependent variables. This lends 
support to the conclusion that pupils in the Multi-Media Bilingual project 
scored significantly better than they probably would have scored if they 
were not in the program. 

Speaking ski 1 1 growth was evaluated through pre-post administration of parts 
II and III of the Philadelphia ESOL Screening Test and through comparison of pro- 
gram participants with students in the city wide Title I ES01- program. The results 
showed gains in the bilingual program similar to those found in the ESOL program . 

A random sample of 43 pupils who were studying ESOL in the context of the 
Individualized Multi-Media Bilingual Education Magnet Model Project were tested 



during the spring of 1978. Their speaking scores were compared to scores at the 
time of initial screening. A t-test for correlated means showed that the gains 
(3.86 on part II of the screening test, 6.51 oh pari III) were highly significant 
(p<.01) . They were virtually the same at the elementary site and at the secondary 
site. 

As these gains could be due to a wide variety of factors, evaluators sought 
to compare the effects of the Title I ESOL and the programs. Drawing upon a 
technique developed for analyzing speaking scores for Title I ESOL, the relationship 
of postscores to prescores (linear, quadratic and cubic terms), program participa- 
tion in 1976-1977, the number of months of English instruction per week, and 
whether the English instruction was provided in the context of the Title VII program 
were explored . 

In addition to the 43 Title VII program participants, a random sample of 57 
Title I ESOL program participants who had also taken pre- and posttests formed 
the subject pool. Previous analyses of Title I ESOL showed that both speaking 
scores increased significantly with the amount of exposure to the program — months 
of participation (p<. 10) and hours of program exposure per week (p<.00l ) . The 
results suggested that participation in Title I ESOL and the Individualized Multi- 
Media Bilingual Education Magnet Mode project were equally effective in increasing 
English speaking ability. AjthougFv students in the Title VI [ sample outperformed 
equivalent pupils in Title I ESOL (by 0.4 points oh the first measure of speaking, 
1 .0 points on the second measure) , this is a chance phenomenon, even when Part 
ll and Part III are viewed in combination. - 

In conclusion, growth of English speaking skill appeared to proceed at a rate 
which was similar to the growth in the Title I ESOL program operating across the 
city . In contrast, the acquisition of comprehension proceeded at a rate superior 
to that found in the comparison program. 

Objective 2: The rate of acquisition of English reading skills, as measured by 
the Wide Range Reading Test, will increase significantly (p<. 05) among pupils 
enrolled in Grade 2 and above. 

This objective was considered attained. The mean score increased at a rate 
greater than that predicted from test-makers norms. The 102 pupils who were 
tested in late December and late May grew 7.39 items; As each item is equivalent 
to one or more months of grade equivalent score, the typical pupil grew at least 
seven months in grade-equivalent score during a five month period. 

The gain observed in the program was statistically significant (t=6.2, df=lbi , 
p<.001) . The item gain was similar at the two sites. At the secondary site, the 
average gain of the 44 students tested during the two periods was 7.72 items. 
At the elementary site, the 58 students grew an average of 7.17 items. 



In evaluating this objective, evaluators could not demonstrate that the observed 
gain In score was greater than would be found in Title I ESQ L, the appropriate 
comparison group, because students in ESOL did not take the Wide Range Reading 
Test. 

The Wide Range Reading Test requires the test administrator to judge whether 
a pupil reads words properly. As many program participants have accents, these 
judgments were difficult. In recognition of this problem, evaluators recommended, 
and project management has accepted , the use of a different test in 1978-1979. 

Objective 3: The rate of acquisition of Hispanic pupils ' reading competence in 
the Spanish language will increase significantly (p<: 05). The tests to be used, 
depending oh grade level are: 

■'Test de desire zas basicas en lectura 

-Pureba de lectura (de Puerto Rico) 

-The InterAmerican Tesi- of Reading (level 4). 

In lieu of measuring rate of acquisition of Spanish, competence of pupils during 
the spring was assessed. Results suggested that elementary pupils performed as 
well or better than pupils in rural Puerto Rico, the secondary level pupils per- 
formed about as well as the 36tn percentile of the tenth grade norm ing group in 
Puerto Rican schools. 

Test de destrezas basicas en lectura was administered in April and May to 
Hispanic pupils at the elementary site. Their performance is shown in Table 1 . 
The mean score for pupils in grade 1 was substantially above the published norms for 
rural Puerto Rico— equivalent to the 95th percentile. In grades 2 and 3 the mean 
scores of the pupils were equivalent to the 50th percentile, indicating that pupils 
in the program performed as well as those in rural Puerto Rico. 

• • 

The same test was also administered to the six program participants in the fourth 
grade because the number of pupils was too small to warrant use of another test, 
even though the test publisher does not provide percentile score equivalents 
for the grade. The fourth grade mean raw score was 13 points higher than found 
in the third grade. Third grade percentile equivalents provide a context for viewing 
the score. The mean score was equivalent to a third grade rural Puerto Rican 
percentile score of 75. 

in the past, many Hispanic high school students have had scores close 
to the guessing level when given tests designed for their grade level. Thus, the 
high school evaluation plan called for use of a test designed for the junior high 
school grades, Prueba Inter-Americana, L4 CEs. Through use of a table of equiva- 
lent scores at adjacent test levels it was possible to estimate a tenth grade percentile 
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rank equivalent. The mean score of the 29 students tested was 45.48, which Was 
equivalent to the 36th percentile of the tenth grade. The raw score standard 
deviation was 21 .24, and the range was from 13 to 92. As most of the students were 
reported to be "ungraded," evaluators did not attempt to divide the high school 
students Into separate grade level groups. 

Objective 4; At least 75% of the 10th grade pupils and 85% of the 1 1th and 12th grade 
pupils enrolled in the program by October, 1977 will complete the school year. 

This objective was attained. Evaluators were unable to describe pupil reten- 
tion on a grade-by-grade basis because many secondary school participants were 
ungraded, but the more stringent criterion stated in tho objective was exceeded 
by all secondary students. 

One hundred and five students of the secondary program were on roll in the earl} 
fall ; Of these, one moved from Philadelphia. Ninety-nine of the remaining 104 
students (95.2%) completed the school year at the secondary site or transferred 
to another school. Twenty-six of these program participants graduated at year 
end. 

Objective 5: Secondary level pupils served by the program will have an absentee 
rate ho greater than that of other pupils in the same grade enrolled in the same 
school. 

This objective was attained, but comparison on a grade-by-grade basis was 
hot possible. The average daily attendance of secondary site program participants 
was substantially better than that of the school as a whdfe. It was also better than 
the average daily attendance of public high schools citywfde; 

As noted in the evaluation of the previous objective, grades were not assigned 
to many secondary program pupils. Evaluators believed! that comparison of parti- 
cipants in the program (regardless of grade) to all students in the school was 
reasonable . 

One hundred and seventy pupiis^parUcipated in the program 
to another school before year-end. These students had an average daily attendance 
of 82. 2%. The Division of Administrative and Survey Research reported that the 
average dally attendance at the secondary school site was 69.7%. As the standard 
error of the average daily attendance of pupils in the program was 1 .5%, evaluators 
are confident that the difference between the school aridflKe program is systematic, 
he., not due to chance. 

Attendance in the bilingual program also compared favorably with the combined 
average daily attendance of all Philadelphia public high schools. The citywide 
attendance was 75.1%, and program students exceeded it by 7.1%. 
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It should be noted that the average attendance of program participants did not 
come from the same source documents as the school and citywlde averages. Evalua- 
tors noted minor flaws in the documents containing program data but could not ^ 
determine whether similar flaws, If any; existed in the others, thus, the differences 
among the average daily attendances should not be regarded as precise. However, 
©valuators have little doubt that attendance was better in the secondary program 
than In the comparison groups. It is unlikely that flaws in the data could have 
accounted for differences as large as the ones observed . 

SUMMARY AND CONCLUSION 

The Individualized Multi-Media Bilingual Education Magnet Model Program 
was well managed and all pupil performance objectives were attained or considered 
attained. 

The program has been in operation for one and one half years. During this 
brief period, project staff and management adapted ideas of the proposals to local 
realities, and brought many disparate elements together to develop bilingual 
programs for multi-ethnic student populations at the elementary and secondary 
levels. 

With the exception of reading test development for non-Hispanic languages, 
enabling objectives were attained. Instructional material and equipment for 
Bilingual Individualized Multi-Media Learning Centers (BIMLCs) were organized, 
students were recruited, students and faculty leaVned how to use the BIMLCs, and 
bilingual education programs serving about 275 pupils came into being. 

English comprehension of students grew at a r.ate which outpaced that of the 
Title I ESOL program, while speaking skill grew at about the same rate as in ESOL. 

Wide Range Reading Test scores grew at a faster rate than predicted from the 
test maker's norms . Hispanic students compared favorably with pupils in Puerto 
Rico. Objectives for retention and attendance of high school pupils were met. 

During the third program year, management and supervisory staff should take 
steps to assure better use of computer-based instruction at the secondary site. 
Stricter observance of the criteria for promotion out of the secondary component is 
also needed . Evaluators plan to improve some instrumentation and firm up the 
experimental designs for some pupil performance objectives . Apart from these 
issues, the program is sound. 
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TABLE 1 



SPANISH READING PERFORMANCE AT THE ELEMENTARY LEVEL 
(TEST DE DESTREZAS BASlCAS EN LECTURA) 



Grade 


Mean 
Score 


No. of 
Stadents 


S tanda rd 
Dev i at i on 


Percenti 
of 


le Equivalent 
Mean* 


1 

2 

3 
4 


67.3 
72.3 
88.3 
101.3 


11 

3 
7 
6 


12.1 
2.5 
13.7 
14.9 




95 

50 n^- 
** 

r 


*Rura1 


Puerto 


Rican Norms, 


end of the 


year. 





**N6 fourth grade norms are published. The percentile equivalent 
of the mean score Is 75 on third grade rural Puerto Rican end-of- 
yea r norms. . 



